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LISTING OF CLAIMS 

THIS LISTING OF CLAIMS WILL REPLACE ALL PRIOR VERSIONS, AND 
LISTINGS OF CLAIMS IN THE APPLICATION. 

L (Previously presented) A method for operating a data classifier, the method comprising: 
providing test input data elements and corresponding test output data elements, 
providing the test input data elements to a data classifier to generate result output 

data elements, 

generating measures of difference based on differences between each test output data 
element and each corresponding result output data element, 

associating the measures of difference with categories corresponding to different 
values of measures of difference, and 

based on the number of measures of difference associated with the categories, 
generating a performance measure of the data classifier. 

2. (Cancelled) 

3. (Previously presented) The method of claim 1, wherein generating a performance 
measure fiirther includes: 

associating the categories with weights, 

for each category, generating a product based on the weight and the number of 
measures of difference associated with the category, and 
generating a sum of the products. 

4. (Previously presented) The method of claim 3, wherein associating the categories 
includes: 

associating the categories with weights based on the values of measures of difference 
associated with the categories. 
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5. (Previously presented) The method of claim 3, wherein associating the categories 
includes: 

associating categories having larger values of measures of difference with greater 
weights than categories having smaller values of measures of difference. 

6. (Original) The method of claim 3, wherein generating a performance measure further 
includes: 

normalizing the sum of the products. 

7. (Original) The method of claim 6, wherein normalizing includes: 

normalizing the simi of the products based on one of: the number of test input data 
elements and a reciprocal of a logarithm of the number of test input data elements. 

8. (Original) The method of claim 1, further comprising: 

training the data classifier, and 

based on comparing the performance measure with a threshold, retraining the data 
classifier. 

9. (Original) The method of claim 1, wherein the test data are based on 
telecommunications data. 

10. (Original) The method of claim 1, wherein the data classifier includes a neural network. 

11. -16. (Cancelled) 

17. (Previously presented) A computer product for operating a data classifier, the computer 
product disposed on a computer-readable medium and comprising instructions to cause a 
processor to: 

receive test data including test input data elements and corresponding test output data 
elements, 
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receive result output data elements generated by the data classifier based on the test 

input data elements, 

generate measures of difference based on differences between each test output data 
element and each corresponding result output data element, 

associate the measures of difference with categories corresponding to different 
values of measures of difference, and 

based on the number of measures of difference associated with the categories, 
generate a performance measure of the data classifier. 

18. (Cancelled) 

19. (Previously presented) The computer product of claim 17, wherein the instructions to 
generate a performance measure further include instructions to cause the processor to: 

associate the categories with weights, 

for each category, generate a product based on the weight and the number of 
measures of difference associated with the category, and 
generate a sum of the products. 

20. (Previously presented) The computer product of claim 19, wherein the instructions to 
generate a performance measure further include instructions to cause the processor to: 

normalize the sum of the products based on one of: the number of test input data 
elements and a reciprocal of a logarithm of the number of test input data elements. 

21. (Previously presented) The computer product of claim 17, further comprising 
instructions to cause the processor to: 

train the data classifier, and 

based on comparing the performance measure with a threshold, retrain the data 
classifier. 

22. (Previously presented) The computer product of claim 17, wherein the test data are 
based on telecommunications data. 
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23. (Previously presented) A system for operating a data classifier, the system comprising: 

an input receiver operable to receive test data, including test input data elements and 
corresponding test output data elements, 

a data classifier operable to generate result output data elements in response to test 
input data elements, 

a difference generator operable to determine a measure of difference between each 
test output data element and each corresponding result output data element, 

a data association manager operable to associate the measures of difference with 
categories corresponding to different values of measures of difference, and 

a performance measure generator operable to generate a performance measure of the 
data classifier based on the number of measures of difference associated with the categories. 

24. (Previously presented) The system of claim 23, wherein the performance measure 
generator is further operable to: 

associate the categories with weights, 

for each category, generate a product based on the weight and the number of 
measures of difference associated with the category, and 
generate a sum of the products. 

25. (Previously presented) The system of claim 24, wherein the performance measure 
generator is further operable to normalize the sum of the products based on one of: the 
number of test input data elements and a reciprocal of a logarithm of the number of test 
input data elements. 

26. (Previously presented) The system of claim 23, the system further comprising a data 
classifier trainer operable to: 

train the data classifier, and 

based on comparing the performance measure with a threshold, retraining the data 
classifier. 
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27. (Previously presented) The system of claim 23, wherein the test data are based on 
telecommunications data. 
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